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This article describes a specific approach to a teacher investigation of students' 
written work. The aim of the investigation was for in-service middle school 
teachers to externalize their thinking about their students' mathematical thinking 
and to then reflect upon these externalizations and make refinements accordingly. 
This aim mirrors the purposes of student-level mathematical activities entitled 
model-eliciting activities. Therefore, the teacher investigation was patterned after 
the design principles for student-level model-eliciting activities. These design 
principles and how the teacher investigation was modeled after these principles are 
described. In addition, the teachers' externalizations, reflections upon, and 
revisions to their thinking about their students' thinking are explained. The article 
concludes with general design principles that other teacher educators can use in 
designing their respective professional development contexts. 


Many of the latest reform efforts in mathematics education stress the 
importance of teachers attending to and understanding their students' 
mathematical thinking (Ball, 1997; Maher & Martino, 1992; NCTM, 2000; 2001). 
However, Ball (1997; 2001) and Schifter (2001) explain that focusing on students' 
thinking in the classroom while maintaining mathematical integrity can prove 
difficult. In addressing such difficulties. Ball (1997) explains that teachers' 
investigations of artifacts of teaching and learning, such as students' written work, 
hold "promise for equipping teachers with the intellectual resources likely to be 
helpful in navigating the uncertainties of interpreting student thinking" (p. 808). 

In this article, a specific approach to an investigation by teachers of their 
students' written work is described. The teacher investigation was conducted as 
part of the on-going Professional Development School collaboration between a 
Midwestern university and a local middle school and was facilitated by the 
author 1 . The aim of the investigation was for in-service middle school teachers to 
externalise their thinking about their students' mathematical thinking and to then 
reflect upon these externalisations and make refinements accordingly. This aim 
mirrors the purposes of student-level mathematical activities entitled model-eliciting 
activities (MEAs) (Lesh, Hoover, Hole, Kelly, & Post, 2000). Therefore, the teacher 
investigation was patterned after the design principles for student-level MEAs. 
This article will begin by explaining the MEA design principles and will then 
explain how the teacher investigation was modeled after these principles. It will 


1 The facilitation coincided with the author's dissertation study which examined the 
teachers' collective interpretations of their students' thinking as a result of participating in 
these investigations of students' work. 
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also describe the teachers' externalisations, reflections upon, and revisions to their 
thinking about their students' mathematical thinking. 

Design Principles for Student-Level Model-Eliciting Activities 

MEAs are designed to help middle school students develop conceptual 
foundations for deeper and higher order ideas in pre-college mathematics (Lesh, et 
al., 2000). Each activity asks students to mathematically interpret a complex real- 
world situation and requires the formation of a mathematical description, 
procedure, or method (i.e., a model) for the purpose of making a decision for a 
realistic client. Because students are producing a model (instead of just a one-word 
or one-number answer) and working in groups, they externalize and reveal their 
thinking throughout the activity and in their final solutions. Such externalisation of 
their thinking assists students in reflecting upon how well their current thinking 
strategies solve the problem and in making appropriate revisions to their solutions. 
MEAs are similar to other mathematical modeling projects that have taken place in 
several parts of the world including, among others, Australia, Denmark, Germany, 
Netherlands, and the United Kingdom (Blum & Niss, 1991). 

The Departing On-Time MEA 2 presents students with departure times for five 
different airlines (see Appendix A). The client, the Ridgewood High School 
Spanish Club, needs help selecting an on-time airline for an upcoming study 
abroad trip. Thus, students have to develop a procedure for ranking the five 
airlines from most likely to least likely to depart on time. The activity is specifically 
designed to help students develop conceptual foundations for statistical concepts 
such as mean, standard deviation, spread, and frequency. 

To develop MEAs, designers rely on six design principles (Lesh et al., 2000). 
The first principle is called the Model Construction Principle, which ensures that 
the activity requires the construction of an explicit description, explanation, or 
procedure for a mathematically significant situation. Such products externalise 
how the students interpret the situation and also reveal the types of mathematical 
quantities, relationships, operations, and patterns that students take into account. 
In the Departing On-Time MEA, students are specifically asked to "Develop a 
procedure for ranking the five airlines from most to least likely to depart on-time." 

The second design principle is the Reality Principle (or the personal 
meaningfulness principle). This principle ensures that students can interpret the 
activity meaningfully from different levels of mathematical ability and general 
knowledge. In the Departing On-Time MEA, students relate to the idea that they 
want to select an airline that is more likely to depart on time, and they quickly 
realise that lower numbers in the table indicate more punctuality. 

The third design principle is the Self-Assessment Principle, which ensures that 
the activity contains criteria the students themselves can identify and use to test 
and revise their current ways of thinking. While working on the Departing On- 
Time MEA, students can return to the data to self-assess whether the results of 


2 It is recommended that readers work through the problem before continuing to read. 
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their calculations seem to reflect what they visually see in the data. For example, 
when students find that the average number of minutes late for the airlines are all 
nearly the same, they return to the data and notice that the airlines do differ in 
their frequency of being late. Such reflections typically lead the students to revise 
their thinking and to count the number of on-time flights for each airline. 

The fourth principle, the Model Documentation Principle, ensures that 
students are required to create some form of documentation that will reveal 
explicitly how they are thinking about the problem situation. Such documentation 
is beneficial for the teacher because it reveals how the students are interpreting the 
given situation. It is also beneficial for the students because it becomes easier for 
the students to visualize and thereby reflect on their thinking. In the Departing On- 
Time MEA, the problem requires students to produce explanations, procedures, or 
descriptions as part of their solution and to explain their solutions in written 
letters. 

The fifth principle is the Construct Share- Ability and Re-Usability Principle, 
which requires students to produce share-able and re-usable solutions. By asking 
the students to produce products that can be used by others beyond the immediate 
situation, students go beyond personal ways of thinking to developing more 
general ways of thinking, often resulting in more powerful mathematics. In the 
Departing On-Time MEA, the students are to develop a ranking procedure to be 
shared with the Spanish Club and to make their procedure general so the Spanish 
Club may use the procedure to rank additional airlines." 

The sixth principle, the Effective Prototype Principle, ensures that the model 
students develop will be as simple as possible yet still mathematically significant. 
The goal is for students to develop solutions that will provide useful prototypes for 
interpreting other similar situations. In the Departing On-Time MEA, the problem 
scenario is clear and simple - find the airlines that are more likely to depart on 
time, and the students' solutions usually provide a useful prototype for 
interpreting other situations in which frequency counts are appropriate. 

The Teacher Investigation 

The purpose of the teacher investigation modeled after these six design 
principles was for the teachers to express, reflect upon, and make revisions to their 
thinking about their students' mathematical thinking. Due to the thought-revealing 
nature of MEAs, the teachers examined samples of their students' work from the 
Departing On-Time MEA . 3 The investigation was centered around the 
development of a Students' Thinking Sheet (STS), which outlines the different 
mathematical approaches used by students in solving the MEA, includes excerpts 
of students' actual work, outlines the mathematics involved in each of the 
students' approaches, and describes the effectiveness of each approach (see 
Appendix B). 


3 Other thought-revealing mathematical activities that elicit multiple solution strategies 
would also suffice for similar teacher investigations. 



Design Principles for Teacher Investigations of Student Work 


55 


The teacher investigation began with an introductory workshop, where the 
teachers worked together to complete the Departing On-Time MEA. Following the 
workshop, each teacher implemented the MEA in his or her classroom. Then, each 
teacher created an individual STS by synthesizing the information in his or her 
observation notes and in the students' work. Finally, at a second teacher workshop, 
the teachers shared their individual STSs and then worked together to create one 
consensus STS. As they created the consensus sheet, they kept their purpose for 
creating the sheet in mind - to explain the students' ways of thinking so that their 
colleagues could later interpret in advance how students typically solve the MEA. 
The Departing On-Time teacher investigation was the second in a series of five 
similar investigations centered around MEAs and was captured through the use of 
video-recording and the collection of teacher products created during the 
investigation. 

Design Principles for the Teacher Investigation 

The purpose of the teacher investigation was for the teachers to express, reflect 
upon, and revise their thinking about their students' mathematical thinking. Thus, 
in designing the investigation, the design principles for student-level MEAs, which 
require students to express, test, and revise their mathematical thinking, were 
mirrored. Also taken into account were teacher-activity suggestions provided by 
Doerr and Lesh (2003). 

For the Model Construction Principle, the teachers should be asked to develop 
a description, explanation, procedure, or justified prediction for interpreting their 
students' work; therefore, the teachers were asked to prepare the STS. Creating this 
sheet required the teachers to formulate an explanation of their students' 
mathematical thinking and served as a prediction of how future students may 
solve the associated MEA. 

For the Reality Principle, the teacher investigation had to a) allow the teachers 
to begin the activity using their existing knowledge and b) require the teachers to 
investigate a realistic situation, i.e. to interpret a situation within the context of 
their own practice. For this investigation, the teachers examined their own 
students' work for the purpose of creating the STS. The teachers were readily 
prepared with their existing knowledge of their students and of mathematics to 
immediately begin interpreting their students' work, and the investigation was 
clearly situated within the context of their practice. 

For the Self-Assessment Principle, the teacher investigation had to ensure that 
the teachers themselves could judge if their interpretations, ideas, or models (such 
as a STS) were meeting their desired purpose. In this teacher investigation, the 
teachers self-assessed their interpretations of their students' thinking by returning 
to the examples of their students' work. Also, they assessed their STS for its 
intended purpose - to capture and explain the students' ways of thinking such that 
their colleagues could later interpret in advance how students typically solve the 
MEA. 

To meet the fourth design principle, the Documentation Principle, the activity 
had to require the teachers to produce documentation that would reveal the 
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teachers' thinking throughout the activity. As a result of engaging in the 
investigation, the teachers created documentation of their thinking about their 
students' thinking as they developed their individual and consensus STSs. Such 
documentation was both in the sheets themselves and in the process of sheet 
design. 

To meet the fifth design principle, the Share- Ability and Re-Usability Principle, 
the investigation had to require the teachers to develop products (such as the STS) 
that were shareable with other teachers or other interested parties and that were 
reusable for themselves, for other teachers, or for other interested parties. The 
teachers met this principle by purposely designing the consensus STS to be shared 
and re-used by their colleagues. Specifically, the teachers believed that their 
colleagues would re-use the STS next year not only to learn in advance how 
students solve the Departing On-Time MEA, but also to evaluate their students' 
solutions and to share with parents at conferences. So, the clients (to parallel 
student-level MEAs) were other middle school mathematics teachers. 

To meet the sixth design principle, the Effective Prototype Principle, the 
activity had to require the teachers to develop a solution that would serve as a 
useful prototype for interpreting other samples of students' work. The teachers 
found that the process of producing the STS (i.e., organizing and classifying the 
students' thinking according to different solution strategies) served as a prototype 
for organizing their interpretations of their students' thinking on successive MEAs. 
By the Departing On-time teacher investigation, the prototype had gelled into a 
typical sequence. First, the teachers took turns sharing the thinking of their 
students on the associated MEA. As the teachers shared, one of the teachers served 
as a recorder and placed the descriptions of the students' thinking in rough 
categories. When the teachers exhausted the different types of students' thinking, 
they returned to the rough categories and made refinements by comparing and 
contrasting the various solution strategies. It was at this point that the teachers 
engaged in the majority of reflecting upon and revising their interpretations of 
their students' thinking. Ultimately, the teachers arrived at three or four main 
categories describing the students' thinking on the respective MEA. With these 
categories delineated, the teachers moved on to discuss the mathematics and the 
effectiveness of each solution strategy to finish the development of their Consensus 
STS. 

Expressing, Reflecting, and Revising 

Modeling the teacher investigation after the student-level design principles did 
lead the teachers to progress through a collaborative process of expressing, 
reflecting upon, and accordingly revising their thinking. Specifically, during the 
Departing On-Time investigation, the teachers began by sharing (expressing) their 
interpretations of how their students thought about and solved the Departing On- 
Time MEA for the purpose of creating the STS as per the Model Construction 
Principle. As the teachers shared their interpretations, one of the teachers recorded 
the students' thinking strategies on a poster sheet to begin meeting the 
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Documentation Principle. Eventually, the teachers had compiled a list of seven 
different strategies: 

1. Find the total number of minutes late for each airline and rank the airlines 
from lowest total to highest total. 

2. Find the percentage of on-time flights for each airline and rank the airlines 
from highest percentage to lowest percentage. 

3. Find the average time late for each airline and rank the airlines from 
lowest average to highest average. 

4. Find the average time late when flights were late for each airline and rank 
the airlines from lowest average to highest average. 

5. Redefine late as consisting of flights that leave more than 5 minutes late. 

6. Count the number of on-time take-offs for each airline and rank the 
airlines from the highest total to the lowest total. 

7. Redefine late as consisting of flights that leave 30 or more minutes late. 

When the teachers felt they had exhausted the different ways of thinking about 
the Departing On-Time MEA, they were prompted to consider their purpose for 
listing these ways of thinking - to create a consensus STS to be shared with their 
colleagues, a purpose prompted by the Share-Ability and Re-usability Principle. 
This consideration encouraged the teachers to begin looking for an easier and more 
precise way to describe the different ways that the students thought about the 
activity. Specifically, the teachers began to decrease their list to three or four main 
ways of thinking. They looked for commonalities and differences between the 
seven strategies (a process of reflecting upon their externalised thinking made 
possible by the Self-Assessment Principle) and subsequently revised their thinking. 

First, the teachers quickly classified strategies 5 and 7 as corresponding to a 
main way of thinking, that of redefining late to be some time period greater than 
zero but less than thirty minutes. Also, the teachers eventually decided that 
students would be likely to calculate the average-number-of-minutes-late-when- 
flights-are-late only after going through the conceptual process of redefining late. 
Therefore, the teachers decided that strategy 4 was a subset of strategy 5 and 7. 

Second, the teachers examined strategies 2 and 6. They realized that finding 
the percentage on-time for each airline was only two computation steps (dividing 
by 30 and multiplying by 100) beyond counting the number of on-time flights for 
each airline. Thus, they determined that the two strategies relied on the same 
underlying concept, that is, an airline with a lower incidence of late flights would 
improve the Spanish Club's chances of catching their connecting flight. The 
teachers decided to therefore combine the two strategies. 

Finally, the teachers compared strategies 1 and 3. They again determined that 
the two strategies relied upon the same idea - airlines with fewer minutes late 
improve the chance of making a connecting flight. The teachers realized that 
finding the average number of minutes late per flight was only one calculation 
(dividing by 30) beyond totaling the number of minutes late. They also recognized 
that the rankings from the total number of minutes late and from the average 
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number of minutes late were the same because all of the airlines contained 30 
flights. 

After revising their interpretations of their students' thinking, the teachers 
arrived at the three main ways of thinking described in their STS: a) finding the 
total or average number of minutes late, b) finding the ratio or percentage of on- 
time flights, and c) redefining late. The teachers arrived at these revisions by 
considering the underlying conceptual idea for each strategy rather than focusing 
on the types of calculations made by the students. Figure 1 contains the teachers' 
poster sheet after they had consolidated similar strategies (see the parenthetical 
remarks at the end of each line describing how the 7 initial strategies were 
combined into the three main ways of thinking). This poster sheet served as the 

Strategies 

1. Total minutes late (see 3) 

2. % "on-time" (see 6) 

3. Average time late (x ^ 30) 

4. Average time late on only late arrivals (see 7) 

5. > 5 minutes late (average, sum) (see 7) 

6. How many "on-time" takeoffs 

7. Define late as greater than or equal to 30 minutes 

model for the STS mentioned previously. 


Figure 1. Teachers’ poster sheet describing the main ways of thinking exhibited 
by their students on the Departing On-Time MEA. 


Conclusion 

The STS investigation led the teachers to express, reflect upon, and revise their 
thinking because it was designed by modeling the student-level MEA design 
principles. The Model Construction and Documentation Principles ensured that 
the teachers externalised and documented their interpretations of their students' 
thinking, thereby making reflection upon and revision to their interpretations more 
likely. The Reality principle, by engaging the teachers in an investigation 
embedded in their own practice, enhanced the teachers' dedication to achieving 
their aim of producing a STS. The Self-Assessment and Share-ability and Re- 
usability Principles prompted the teachers to revise their interpretations of their 
students' thinking. In particular, the teachers looked beyond computation and 
calculation differences in their students' thinking to take into account the 
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underlying concepts. The ability to identify the concepts with which students are 
working is crucial as it assists teachers with making instructional decisions based 
on their students' current understandings. Finally, as a result of the Effective 
Prototype Principle, the teachers developed a prototype for organizing, classifying, 
and making sense of their students' thinking on thought-revealing mathematical 
activities. Such a prototype assists teachers with interpreting their students' 
thinking on various mathematical activities. 

The teachers also learned that students can approach tasks such as MEAs using 
different ways of thinking and that not all mathematical tasks have just one "right 
answer". As the teachers reported in a handbook about MEAs, " 

MEAs do not typically have one right answer; however, some solutions are better 
than others. Thus, when assessing your students' work or providing feedback to 
your students, it is more helpful to ask, 'How well does this particular solution 
meet the needs of the client?' instead of asking, 'Is this the right answer?' 

Patterning the teacher design principles after the student-level MEA design 
principles proved fruitful for this investigation, both for encouraging teacher 
development and providing documentation about teacher development. As such, 
other teacher educators will likely find these principles useful in their professional 
development contexts. In the meantime, research is necessary to understand more 
fully just how engaging teachers in such investigations may impact their abilities 
to make sense of their students' thinking within the classroom and whether such 
investigations help teachers deal with the complexities identified by Ball (1997; 
2001) and Schifter (2001). As we await such research, Table 1 provides questions 
for each design principle that teacher educators can consider in planning their 
respective teacher investigations and professional development. 
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Table 1 

Verification Questions for the Six Design Principles for Teacher 
Investigations 


Construction 

Principle 

Reality 

Principle 


Self- 

Assessment 

Principle 


Documentation 

Principle 

Share- Ability 
and Re- 
Usability 
Principle 

Effective 

Prototype 

Principle 


Are teachers required to make constructions, explanations, 
assessments, and/or decisions about the students' work and 
thinking? 

Are teachers asked to make sense of the students' work and 
thinking based on extensions of their own personal knowledge and 
experiences? 

Does the task require teachers to interpret situations within the 
context of their actual practice? 

Is the purpose clear so teachers themselves can judge if their 
interpretations are bringing about desired consequences in 
particular contexts? (Doerr & Lesh, 2002) 

Will the teachers be able to judge for themselves when they need to 
revise or extend their interpretations of the students' work and 
thinking? 

Will completing the task require teachers to document how they 
are thinking about the students' work and thinking? 

Are teachers required to develop products that can be shared with 
other teachers and researchers and that can be re-used by 
themselves, other teachers, or other interested parties? 


Does the solution provide a useful prototype for interpreting other 
students' work and thinking? 
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Appendix A: Departing On-Time Model-Eliciting Activity 

In June, Ridgewood High School's Spanish club is going on a study abroad trip 
to Venezuela. Last year, when traveling to Barcelona, their connecting flight to 
Reykjavik, Iceland was late. They missed their connecting flight and had to stay 
overnight in the airport! This year the club is being more careful. So far, they have 
identified five airlines that fly from O'Hare Airport to Venezuela, but they are still 
identifying more airlines. The flights all have a connecting flight in Mexico City. 
The students want the airline with the smallest chance of departing late from 
O' Hare so they can catch their connecting flight. Below is information about 
departure times for flights leaving O' Hare and arriving in Mexico City for each of 
the five airlines. Develop a procedure for ranking the five airlines from most to 
least likely to depart on-time. Describe your procedure in a letter to the Spanish 
Club so they may use your procedure with additional airlines. 


Sky Voyage 
Airline 

Central American 
Airline 

Mexico 

Express 

Sudamerica 

Internacional 

Southeast 

Airline 

5 

15 

9 

0 

0 

0 

9 

5 

25 

5 

20 

4 

5 

0 

0 

5 

0 

5 

9 

9 

0 

0 

125 

0 

40 

6 

14 

10 

0 

0 

0 

20 

5 

4 

5 

0 

15 

10 

0 

25 

15 

16 

0 

35 

10 

0 

0 

4 

0 

30 

0 

0 

10 

0 

12 

7 

15 

10 

10 

0 

0 

10 

10 

5 

0 

5 

10 

9 

55 

10 

40 

25 

7 

0 

9 

4 

5 

12 

0 

5 

0 

20 

5 

0 

0 

0 

15 

0 

17 

27 

0 

11 

10 

5 

11 

0 

12 

7 

0 

0 

3 

0 

13 

65 

30 

60 

5 

0 

5 

5 

5 

0 

0 

0 

0 

0 

30 

10 

0 

4 

7 

4 

5 

2 

40 

0 

5 

4 

0 

0 

0 

10 

6 

0 

15 

123 

10 

5 

75 

0 

0 

25 

7 

0 

6 
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Appendix B: Students' Thinking Sheet for the Departing On- 
Time Model-Eliciting Activity 


Description of Solution Strategies Mathematics Effectiveness 


Strategy #1: Students may find the total 
number of minutes late for each airline and 
then find the average number of minutes 
late for each airline. Finally, they rank the 
airlines from lowest to highest 


We have been looking over various Airlines' 
flight times from the month of June in 1 999. 
We added up the total amount of minutes 
they were late and divided it by 30 to find the 
average amount of minutes a particular 
airline was late per day. 

Average Number of Minutes Late: 


Adding the minutes; 
finding the average 
number of minutes; 
comparing and 

ordering the averages 
or total minutes to 
rank the airlines; and 
rounding the averages 
to the nearest tenth. 


Easy to implement, 
but not very effective. 
A few very late 

departures by one 

airline will be equal to 
several minimally late 
departures by another 
airline. Also, the 

averages only differ 

by a few seconds. 


Strategy #2: Students may count the 
number of times that each airline is on-time 
(count the zeros). Then, they may find 
either the ratio of on-time flights or the 
percentage of on-time flights for each 
airline. Finally, they rank the airlines from 
highest to lowest. 

We found our answer by counting how 
many times the airlines were on time. We 
picked the one that was on time the most. 

Counting Number of On-Time Flights: 


Adding the number of 
on-time flights; 

finding the ratio of on- 
time to total 
departures; finding 
the on-time 

percentage; comparing 
and ordering to rank 
the airlines. 


Easy to implement 
and more effective 
than strategy #1, but it 
does not take into 
account that departing 
up to 20 minutes late 
may still be okay for 
catching a connecting 
flight. 
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Strategy #3: Students may redefine late as 
occurring when a flight departs more than 
5-20 minutes late and incorporate this new 
definition into strategy #1 or strategy #2. 
For example, they may find the average 
number of minutes flights are late when 
they are late, or they may count the number 
of times that each airline departs within 5- 
20 minutes or less. 

Counting Number of Flights Departing within 

If you are wanting to know the probability of 
the airlines being late, you must find the ratio. 
We did this by not counting 5 minutes and 
under as being late. Sky Voyage Airline’s 
ratio of being late, for instance, is 22/30, 22 
being the times the airlines is not late and 30 
being the amount of days. 

Fewer than 5 Minutes: 


Determining 

an 

A little more complex 

acceptable range 

for 

to implement 

and 

departure times; same 

more effective 

than 

mathematics 

as 

strategy #1 or 

#2 

strategy #1 or #2. 


because it allows 

for 



minimally 

late 



departures. 

Its 



effectiveness depends 



on identifying 

an 



acceptable range 

for 


departure. 



